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ABSTRACT 
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building, including the emphasis on professional development. It states that 
despite self-reports and surveys, scant, if any, replicable empirical 
evidence relates professional development to changes in teacher behavior and 
improved student outcomes. The paper calls for educators to critique the 
assumptions and assertions behind the call for professional development and 
offers some available scientifically based research approaches to education 
improvement. The paper goes on to support more scientifically based research 
in education and presents a new model for professional development based on 
scientifically based research. Included are: a sample of studies that support 
professional development, but fail to find empirical evidence of teacher 
change or student improvement and appendices that contain: a glossary of 
terms, a checklist for staff development, a survey instrument used to collect 
data on research-based information that improves student outcomes, and a six- 
level phase model to help policymakers and practitioners sort the issues in 
educational research; and 30 references. (WFA) 
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Education Administration (Ed Ad) and Capacity for School Improvement.' 

Thoughts About Education Administration And Improvement 

We’ve all heard it: “Think out of the box”. This adjuration accompanies most any 
serious consideration of education improvement. If “the box” is so debilitating that we must 
always think out of it to improve, it is time to think seriously about the box. My often 
contrarian ideas about education administration (Ed Ad) have usually been met with polite 
“Ho Hum.” Now, I’ve added a strange ally to my constant call for EdAd to focus on schools 
and make schools better places for kids. The ally is scientifically based research (SBR): The 
“strange” is that SBR is, in my mind, about the only useful part of the No Child Left Behind 
(NCLB) Act. 

Since 1967, I’ve collected bits and pieces about Ed Ad specifically and education 
improvement efforts in general trying to determine why, over the years, education reform has 
been cyclical and mostly stagnant. So here’s one more research-based challenge to 
traditional shibboleths. The task of explaining these views about school improvement, 

EdAd, and SBR seems as daunting as explaining three dimensions to a native of Flatland. 
(Abbott, 1952) 

The snare is set early and easily. Capacity building is one process for education 
improvement. Calls for professional development (PD) or staff development (SD) or 
inservice programs (etc.) are usually high on the list of capacity-building stratagems. In a 
field whose core business includes instruction and in which many practitioners hold titles 
such as staff developer or human resource consultant, calls for more PD or SD as a primary 
capacity-building tool for improvement are seldom questioned. Many professors have an 
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inservice moonlighting business that may expand dramatically with the 2001 reauthorization 
of the Elementary and Secondary Education Act of 1965 (PL 89-10), mislabeled as the No 
Child Left Behind (NCLB) Act^ which recommends lots of SD and funds it through Title II. 

The NCLB is, ironically, the same act that describes “scientifically based research” 
(or SBR) and mentions more than 100 times SBR and the need to base education 
improvement on SBR results (e.g., Slavin, 2002; Feuer, Towne & Shavelson, 2002). This 
paper offers a view contrary to most NCLB pronouncements, excepting the call for SBR as a 
cornerstone for education improvement. 

The idea is formulated as a problem in a discrepancy model, showing dissonance 
between the present empirical state and some desired normative condition (Achilles, et al., 
1997; Haller & Kleine, 2001). The unsupported aura of education failure is pervasive so any 
proposed solution is potentially contentious. A problem, however, allows for analysis and 
consideration of possible solutions that are administratively mutable: Competent people can 
do something and assess the changes. 

The Problem 

The constant focus upon “Professional Development” or PD, nearly to the exclusion 
of other improvement strategies has derailed education improvement. This focUs wrongly 
blames teachers and (indirectly) teacher preparation for presumed education deficiencies and 
shields other possible causes. DISCLAIMER : This paper is not an attack on PD, but it does 
raise questions about various approaches to capacity building. 

Definitions are important in research and in discourse. Key terms are defined in 
Appendix A, Glossary. PD as used here excludes advanced degree work. However, an 




3 



AERA/Capacity 



unpopular study by Haller, Brent, and McNamara (1997) called into question the value of 
advanced preparation in Ed Ad if preparation is continued in traditional ways. 

Selected Examples and Empirical Evidence . Teachers seldom clamor for more 
“Professional Development” or PD. Regardless of self-reports and surveys, scant, if any, 
replicable, empirical evidence (SBR) relates PD to a) changes in teacher behavior and b) 
improved student outcomes. Many articles and ideas for improvement perpetuate PD with 
unsupported assertions that PD is required. However, many studies that have demonstrated 
improved outcomes and included and assessed a PD component [e.g. Student Teacher 
Achievement Ratio or STAR (Achilles, 1999), class-size reduction or CSR in California, and 
other class-size studies] do not find observable teacher changes even if teachers self-report 
them. The absence of such a finding is particularly notable in relation to organization 
changes, such as class size, that dp provide replicable student gains. In summary, PD has 
received huge expenditures of time, funds, and effort: PD is touted in NCLB that argues for 
scientifically based research or SBR. One would expect demonstrable, replicable SBR 
evidence of successes as a base for the NCLB support of PD, but none is presented. The 
depth and ubiquity of the problem are evident in the question: “Can you provide two or more 
high quality, replicable, empirical studies (SBR) of the positive effects of PD on teacher 
behavior and especially of its effects on student short and long-term success as usually 
measured?” Table 1 offers some support for the empirical claim. 

TABLE 1 ABOUT HERE 



Normative Claim. The continuing and probable NCLB-driven large investments of 
time, funds, and efforts in PD should both be based upon and provide SBR evidence of 




4 



AERA/Capacity 



changed teaching that can be linked directly to improvements in student outcomes. A 
yawning abyss exists between the available empirical evidence and the normative claim. 

The problem seems to have two dimensions: 1) To find an answer to the compound 
question presented above through both an extensive interpretive research review and 
- conducting such SBR studies, or 2) To answer the question, “ Why does there seem to be 
minimal or no evidence that PD has the anticipated demonstrable effects?” A third, and more 
controversial possibility is presented here. Critique the assumptions and assertions behind 
the omnipresent and SBR unsubstantiated call for PD and offer some available SBR 
approaches to education improvement. 

Logic for and Challenge to Staff Development As Presently Done for Capacity Building ^ 
Constantly renewing one’s knowledge, skills, and attitudes requires personal and 
professional attention. In some fields (e.g., medicine, dentistry, CPAs) the impetus for PD 
comes from and is primarily supported by the person. In education, however, SD and PD are 
often mandated with specific days in the school calendar allocated as part of the educator’s 
work year. Typically, students do not attend school on these days. 

A huge commodification for PD has emerged, with a deeply embedded infrastructure, 
often manifest as entire departments in central offices dedicated to PD and Human 
Resources, and featuring PD experts in any fad. PD is driven by a national organization, a 
thriving textbook market, cadres of staff developers, and hordes of special projects — many 
untested and unproven as to student benefits. Some projects promise to address any actual or 
perceived need — ^but only after the project’s developer has given potential users ample 
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inservice exposure. Federal, state, and local funds fuel much PD that seems top-heavy and 
out-of-balance when assessed against three criteria. 

1 . Those who receive the PD will demonstrate positive change in skills, knowledge, 
attitudes, and behaviors. 

2. The desired changes and improvements should be measurable and observable in 
the short term and in the long term: They become the norm until better 
knowledge skills (etc.) are available. 

3. Because the ultimate beneficiary of PD should include the clients (i.e., students) 
the results of PD should lead directly to observable, measurable positive change 
in student outcomes on clearly defined criteria. 

Although the National Staff Development Council has advanced the knowledge about 
and sophistication surrounding PD (e.g.. Sparks & Loucks-Horsley, 1989; Showers & Joyce, 
1996; Wood & McQuarrie, 1999; Wood & Thompson, 1993), many questions still require 
answers. Before they continue to accept the value of PD as capacity building and to invest 
heavily in it — ^policy persons should require data to support the investment. The time is long 
past when those who assert that PD improves teaching, and ultimately student performance 
should provide replicable evidence to support their claims. While waiting for PD evidence, 
its time for action using SBR evidence . 

Consider One SBR Alternative: Change The Box 

Sparks (1993) noted that “...quality improvement expert W. Edwards Deming 
estirnates that 85% of barriers to improvement reside in the organization’s structure and 
-processes, not in the performance of individuals” (p. 3). Intriguing. Think about it. If this is 
correct, then PD for workers could only get at 15% of the barriers to improvement even if the 
PD were 1 00% successful (improbable)! Now is the time not just to “think out of the box” 
(e.g., to tinker with PD), but to change the box. Improve the organization for education as a 
SBR approach to capacity building. 
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In his last years, Deming revised his 85 - 15 estimate of the problem. With the help 
of graduate students, I pursued this idea. On December 5, 1993 the front page of the 
Business Section of the Los Angeles Times ran . an article quoting Deming at a southern 
California seminar, “All that happens comes from the system, not the workers. It’s 
absolutely frightening, just frightening.” This idea appears in Deming’s (1993) book. The 
New Economics (Second edition, 2000): “In my experience, most troubles and most 
possibilities for improvement add up to proportions something like this: 

94% belong to the system (the responsibility of management) 

6% are attributable to special causes... 

No amount of care or skill in workmanship can overcome fundamental faults of the system” 
(pp. 33-34). If Deming is correct . . . ? . 

Scenario. Pat’s pre-service teacher preparation included training in miming records 
(RR) to assess and assist beginning readers and appropriate use of portfolios as a way to 
monitor student-growth. Pat demonstrated solid knowledge of these skills and used them 
satisfactorily in student teaching, usually while the regular teacher helped students with 
whom Pat was not working. The classroom teacher praised Pat’s work. 

After employment, Pat’s first-grade class, like others in the building had 29 students 
considered “about average” with a couple of “included” students. Pat’s plans called for RR 
(each child observed weekly), portfolio assessment that would involve parents, and 
individualized work emphasizing “the basics” of language and numbers. 

Pat soon found that there was not time for RR and portfolios. These and other skills 
learned in teacher preparation soon faded in favor of large-group instmction, rote, testing, 
worksheets, and classroom management: Pat learned crowd control. Colleagues told Pat to 
“forget college idealism.” “Welcome to the real world.” 

An esteemed district supervisor returned from a conference enthusiastic about RR and 
sold the idea to the administrative council, which mandated RR training for all K-3 teachers. 
Pat dutifully attended 20 hours of training and demonstrated the use of RR, including 
diagnosing a student’s needs and re-teaching. When the supervisor conducted a follow-up 
visit, she wrote an unfavorable observation indicating Pat’s non-use of RR, a “scene” 
repeated in other K-3 classrooms. The supervisor reported to the administrative council: 

“We brought the K-3 teachers the most up-to-date tools, and even after extensive PD with 
follow-up, the teachers won’t use what we required. No wonder we can’t improve!” 
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Unfortunately, the above scenario is not uncommon. More unfortunately, there are 
great alternatives. Administrators seem not to know them or to try them. 

One Capacity for School Improvement 

Given a) W. Edwards Deming’s claim that problems in organizations are directly 
connected to an organization’s structure and management (85%-94%) and that only a few are 
related to personnel (workers), b) the continuing pervasive finding of little demonstrable 
student gain or teacher- change from PD, and c) an espoused interest for improving education, 
it is time to stop exhorting people to “t hi nk out of the box” to improve education. If it is the 
box, change the box. Start by making organization adjustments aligned with the findings of 
“good” research (SBR). Changes should be in concert with useful, time-tested theories and 
consensually validated exemplary practices or informed professional judgment (IPJ) of 
outstanding teachers. 

The evolving list in Table 2 awaits additions of SBR (demonstrated) “things” that 
advance the efficiency and effectiveness of EdAd, improve schooling outcomes for students 
and that evaluate, expand, and advance the field’s knowledge base. Exclude “Programs” that 
are for sale unless they meet SBR, theory, and IPJ tests. Additions to the list should be 
products of SBR, of theory, and of IPJ; be administratively mutable; and improve student 
achievement in at least four categories: The A, B, C, D’s or Abecedarian Compact: 

(Achilles, 1999) is similar to Comer’s desiderata for school as described by Haynes and 
Emmons (1997). Student achievement is positive in: 

Academics, as shown by test scores and other indicators 
Behavior and discipline in and out of school 
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Citizenship and participation in and outside of school 

Development into productive, contributing, humane adults with constructive 
self-concepts 

The emerging list in Table 2 offers SBR-supported education-improvement ideas. According 
to Deming’s theory, administrators are responsible for these changes. 

Appendix B offers a summary of points presented here. 

^ TABLE 2 ABOUT HERE 



A New Model for Professional Development 

Educators often claim that education is a profession similar to law, medicine, clergy, 
etc. Not to know SBR and replicable, empirical evidence of ways to improve the field while 
relying nearly exclusively on as-yet unsubstantiated traditions would seem to defy 
“professionalism.” How do Deming’s ideas hold up in a profession such as medicine, a field 
that does not have state-mandated PD or inservice days? 

The New York Times Magazine (March 16, 2003) carried a series on medicine and 
medical education. One MD (Sanders, 2003) reflected on the Dean at her medical school 
saying “. . . half of what we teach you here is wrong — unfortunately, we don’t know which 
half’ (p. 29). Another MD noted: “A 1999 Institute of Medicine report estimated that 98,000 
deaths occur in the United States every year because of medical errors. Most of these deaths, 
the report said, do not occur because of individual mistakes but because of flaws in the way 
hospitals and clinics operate” ( Jauhar, 2003 p. 35). Citing a tragic transplant issue, the 
doctor reported that “the fault was not in any one individual but rather in the structure of 
care. . .” (p. 35). Later in the same series Burton (2003) noted that “Five percent of doctors 
are said to be responsible for more than half of malpractice post-trial payouts” (p. 48). The 
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medicine scenario sounds eerily like the Deming 85%-94% contention and the present 
arguments against PD as the primary way to “improve” education outcomes. 

Because it is important for professionals to sharpen old skills and learn new 
knowledge and skills, PD is needed. It will assume a secondary but professional stance. 
Because of the dearth of SBR evidence that PD provides teacher change and student 
improvements in education, PD should follow the model set by fields such as medicine and 
law and rely on user (e.g., teacher) choice in a market model: Teachers will select their PD 
needs and delivery system. They will pay for PD directly to the provider. There will be 
minimum state or district payout for “mandated” SD days, although there may be required 
inservice days to implement policy or legal mandates. The PD timeframe would usually 
occur so that practitioners do not lose time instructing the students for whom they are 
responsible. Administrators, not teachers, will receive PD that demonstrates SBR 
organization change that fosters school improvement. 

With the crutch of “more PD” removed, principals — the administrators and top 
management of schools — now become responsible for much of Deming’s 85%-94% of the 
barriers to improvement. (Did the call for PD and its easy out for principals derail the 
organization idea of school restructuring?). The non-principal portion of the 85%-94% rests 
with administrative levels above the school; district, state, federal. What a Pandora’s Box 
this promises to be. 

Examples of teachers successfully improving their practice in demonstrable ways 
reflect the processes of staff reviews of difficult medical issues (teacher conversations about 
teaching, teacher study groups, etc.); they sound not at all like producer-driven “inservice 
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sessions” or SD mandated by bureaucrats (who don’t attend them), paid for by public funds, 
mocked by teachers, and that take teachers from classrooms. 

Denouement: Bits and Pieces of a Kaleidoscope 

Preparation . Education occurs mostly in schools so the practice of education reform 
starts there. Programs that prepare principals will need to be grounded in SBR principles. 
Programs should assure that principals-in-waiting can assess against SBR standards and 
theory the claims made for research and products built upon the research. The EdAd students 
must be comfortable in the habit and practice of critique: What is “good” research? How do 
competing claims stack up? How can educators use (implement) and assess the research? 
Curricula and testing in EdAd preparation would include knowledge of SBR results that help 
students, critical thinking and critique skills, action research and evaluation methods. 

This preparation component reflects part of the “what” leg of the triad of things a 
competent EdAd person must know: What to do (e.g., the foundational education 
knowledge). How to get the what done (Leadership), and Why (moral, legal, policy, ethical 
reasons) something should or should not be done. [See Achilles, C. M. & Price, W. J. (2001, 
Winter), “What is missing in the current debate about education administration standards.” 
AAS A Professor . 24 (2), 8-14]. These three categories (What, How, Why) are logical ways 
to classify, deliver, and evaluate EdAd preparation and performance. 

To advance the field beyond the morass its critics claim for education and EdAd, 
many levels of action are required. A first step— perhaps the most difficult — is to seek 
consensus on the potential of SBR outcomes to help improve schooling and to establish some 
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priority for their use. Next, this consensus must be communicated clearly and carefully 
within the education community and to the involved publics: parents, citizens, policy 
persons, politicians. Clear communication will help restore public confidence in public 
education as a foundation institution of society, like health and religion. 

These steps will exude a clarity of purpose in the application of student-centered, 
consensus-driven education improvement steps built upon valid knowledge sharpened by the 
habit of professional critique. (Try to make rationale what has become unraveled in recent 
years with the attack on public education.) Schools and children certainly are not the sole 
responsible agents for social action to address such challenges as poverty, diversity, and 
economic decline that fill the media. Educators’ primary focus is helping mostly young 
clients who don’t yet vote obtain the knowledge, civility, and skills to become “solid 
citizens” capable of informed, ethical action in social settings. 

As characteristics of the clients change, education must adapt and society must 
acknowledge the changes. (Achilles & Mitchel, 2001). Armed with certitude about the 
validity of knowledge underlying education, EdAd persons must build the capacity to 
communicate clearly what actions they plan to take, how they will proceed, and why the 
selected actions are required and fitting. Focused research, judicious trials and evaluations, 
and rigorous professional critique will replace fads, quick fixes, and expediency that 
presently clutter the path of education improvement. These steps won’t be easy. 

Educators have not scored well recently when given opportunities to apply research to 
guide education improvement. Consider two examples dealing with class size. The 
California CSR was a textbook example of good intentions implemented totally contrary to 
the extensive research. Florida voters (2002) approved a Constitutional Amendment for 
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class-size adjustments that the Florida Association of District School Superintendents 
(FADSS!)) opposed in a formal “white paper” (8/02). When faced with implementing the 
mandated small classes, the administrators supported politically expedient “Pupil-Teacher 
Ratio” or (PTR) steps that research has shown do not improve overall schooling outcomes; 
legislators recessed without acting on the amendment. 

Standards for admission to EdAd and for preparation of those admitted need 
attention. After reviewing the trend of decline in one measure of cognitive ability of EA 
candidates (the GRE), Keedy and Achilles (2001) mused about the source of the intellectual 
firepower needed for EdAd’ s next challenges. Those circling the wagons around present 
preparation programs might heed medicine’s humility; “Half of what we teach you re here is 
wrong — unfortunately, we don’t know which half’ (Sanders, 2003, p. 29). We (EdAd) might 
consider checking this idea carefully using our programs of study and course syllabi . I’ve 
provided a couple examples in Appendices C and D. 

Appendix C includes a questionnaire that I often give to practicing EdAd persons 
when I make presentations. I have given and reviewed more than 2000 of these. The four 
sheets in Appendix C are typical summaries of the responses. 

Appendix D offers ways an EdAd person might assess if some “education 
improvement” idea meets reasonable tests of valid, reliable, replicable SBR, or if it is a fad. 
Each example in Appendix D (where applicable) uses as the education improvement idea the 
considerable research on class size. Remember; Ask a person promulgating a program, 

“Can you provide two (or more) empirical, replicable (SBR) independent studies that show 
positive short and long-term improvements (A, B, C, D) for student outcomes?” Enjoy! 
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Table 1. Sample of Studies that Support Professional Development (PD) but Fail to Find 
. Empirical Evidence of Teacher Change or Student-Improvement Gains. 



Source/Focus/Desien 
Guskey, T. R. (1997) 

Haller, Brent & McNamara (1997) 
Advanced training in EdAd, 

Carpenter (2000) reviewed school reform 
efforts (1990-2000). 

Newmann et al. AERJ (2000) and 
American J. of Education . Broadly 
theoretic and conceptual paper. School 
capacity focus. 



Garet et al. AERJ (2001). Focus on 
teacher learning. 

Covert, S. (2003) Review of research for 
a dissertation on PD. (E. Michigan U.) 



Tienken, C. (2003). Review of research 
for a dissertation on PD (Seton Hall 
University). 

Guskey, T. R. (2003). Review of 13 lists 
of “effective PD.” 



Design/Outcome/Ouote 

“Research needs to link PD and student 
learning.” 

No measurable difference on “Effective 
Schools” indicators of advanced 
training. 

PD (and other reforms) have had little 
impact on student achievement or on 
school improvement. 

“The case for substantial investment in 
[PD] is vulnerable because of an 
absence of research that links specific 
forms of [PD] to changes in teacher 
learning and practice and to student 
achievement gains. . .” (p. 53). 

National probability sample (n=1027). 
Teacher self-reported info, on a survey. 
No evidence of student gains. 

“Given the lack of studies which 
demonstrate PD effects on teachers or 
student outcomes. . .” (p. 25). Argues 
for attention to theory to guide PD. 

Minimal evidence of PD changing 
teacher behavior or student 
achievement. Job-embedded PD seems 
promising. 

“But that research includes rigorous 
investigations of (PD) and 
improvements in instructional practice 
or student outcomes.” (p. 749) 
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Table 2. An Emerging List of SBR Results that Influence Schooling Outcomes Positively on 
the ABCD Dimensions: Change the Box 

Instructions: “Brainstorm” ideas onto the list. Refine them later. Provide a source 

(i.e., author, study) or research for an idea (e.g. class size, K-3 Project STAR). Estimate cost 

as high, moderate, medium, or low. (References available). 

A. Organization. Not PD 

1 . Appropriate-size learning groups for the task. Class size, if the class is the unit of 
organization for instruction. STAR, K-3; SAGE; Many Studies. (Achilles; 1999). Low 
cost if done in accordance with the research. 

2. Attention to school size (K. Cotton, Fowler & Walberg, V. Lee), such as Learning 
Communities, Academies, School Within A School, etc. (Low Cost) 

3. Tutoring (Bloom’s 1984 2-Sigma problem) (Reading Recovery) (High Cost) 

4. Transitions (Elementary-Mid/Mid-HS). (Low/Moderate Costs). (See also #2, 17) (J. 
King-Rice, S. H. Achilles, M. Wilson,) 

5. Retention in grade (Harvey; STAR; Holmes & Matthews, Shepard & Smith; etc.) (Low 
costs). Actually a considerable cost savings if reduced). 

6. Non-graded or multi-graded schools (especially work in Australia; Pavan; Goodlad & 
Anderson); (Low cost). See #7. 

7. Multi-age (Australian studies: Pavan; Goodlad, & Anderson) (Low Cost) See #6. 

8. Quality Pre-K (Perry Pre-School and its longitudinal findings.) (Moderate/High cost but 
positive, cost-effective returns if well done). (Barnett, Head Start, Schweinhart & 
Weikart) 

9. Cohorts, The Cohort Effect: Most research is on adults. STAR. Morrill (2003) (Low to 
no cost). 

10. Looping. Little solid data yet, but small studies. (Low cost). See #6, 7, 11. 

11. The Class Teacher (Denmark). See Morrill, Kappan , Feb. 03, (Low cost, high outcome) 
(A type of looping). 

12. “Hands-on” learning opportunities (Use of time and space). (Low cost). 

13. Use of Time in Schools (National Commission on Time and Learning, 1994; Goodlad, 
1984; Time on Task Studies). (Low cost). Recapture wasted time in schools: “Prisoners 
ofTime”. 
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14. Participation (Cooperation, Involvement, Engagement etc.) (Finn; Voelkl; Rumberger; 
Lindsey). STAR. (Moderate/Low cost). 

15. Relationships (J. Comer, et al.) (Moderate/High costs, depending on methods). 

16. Herzberg’s “Motivators”; “Motivation” Research (Maslow, Herzberg) (Costs?) 

17. Effective Schools (Edmonds, Lezotte, etc.) (Low/Moderate costs) 

18. “Community/Family” (Bateman, Small-class Research, etc.) (Low costs) 

19. Coherence (Newmann, Bryk, et al). Seamless In-class Transitions (SSS). (Low costs) 

20. Use of Teacher Assistants (Para-pro); (Gerber, et al, Finn, et al, Achilles, STAR 
“Sense” Papers, Haberman, Title I (Minimum time and use in classrooms). (Low cost or 
cost savings). 

21. “Structures” Studies: (Hoy, Sweetland). (Low/Moderate costs). 

22. Capacity Building. Spillane, others. (Low cost). 

23. Theories of Practice: (Keedy & Achilles). (Low or no cost). 

24. Cooperative Learning (Slavin; Johnson & Johnson, etc.). (Low/Moderate costs). 

25. Facilities, Air Quality, “Sick Building” Syndrome. (J. Prout). (Cost could be high). 

26. Ways to address Student Mobility (e.g., migrants, etc.) (Costs?) 

27. Full-Day Kindergarten (in a small class). (STAR, “Test Score Value of K”; Barnett’s 
work with head Start). (Moderate/Medium Cost, but cost effective) See Slavin, Wasik, 
etc. Early Prevention of School Failure. Head Start Studies, Krueger. 

28. Delivery Model (Planned, Coherent vs. Fragmented, “Pull-Out”) (STAR, SSS, Newmann 
et al., B. Taylor in CMA Monograph, and notes to AASA (2003) pages 4-5) (See Title I 
Critique) (Low/Moderate costs). 

29. Parent/Community Involvement. (Epstein, Hoover-Dempsey, Ho & Willms, etc.) (Low 
cost). 

30. Teacher Use. Teaching, Redeployment, The National Commission on Teaching (D. 
Hammond, 1968, p. 6), Sharp (2002). 

• High quality, qualified teacher in every classroom; 

• What teachers know and can do is one of the most important influences on what 



students learn.... 
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. School reform cannot succeed unless it focuses on creating the conditions in which 
teachers can teach and teach well . (Emphasis Added). (Cost?) 

31. Reconfigurations of Time in School: Block, Copemican, Year-around, Extended day 
and/or year. Canady & Rettig; Carroll. (Moderate Cost). 

B. Curricular and Instructional Issues Supported by Organization Changes 

. Developmentally Appropriate Schooling (Piaget; Feuerstein; V ygotoky; Dewey; 
Perry Pre-School)) 

• Homework, Appropriate uses of; (Cooper, H.) (Low cost) 

• Teaching/Leaming styles (Dunn & Dunn; Campbell; Gardner) (Low cost). 

. Problem-Based Learning; Projects, Service and Community Learning. (Low cost). 



C. Personnel Focus (New PD Forms) 

. Job-Embedded PD (Tienken, 2003; Caufield-Sloan, 2001). Low cost. 
• Teacher “Conversation” (See Author note 3). Low cost. 
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Glossary of Important Terms * 
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Inservice Training refers to one-time or short-term training, usually a specific 
workshop or large-group session to present information or a basic skill, easily learned, 
usually delivered primarily via one-way communication. The training may include j ob- 
embedded elements, but there typically is no follow-up. 

Job-Embedded Staff or Professional Development is planned and continuous 
training that in education specifically emphasizes teaching or instruction skills and 
knowledge related to student outcomes. Examples include action research and evaluation, 
structured study groups, peer coaching, mentoring. 

Professional Development (PD) is on-going, planned, continuing education through 
which certified, qualified teachers and other education professionals improve skills, 
knowledge, and attitudes/dispositions related to assisting clients (e.g., students) achieve 
goals of tlie organization (i.e., improved student performance and outcomes). A primary 
interest is to improve the professional’s long-term value in workplace performance. 
Interaction and two-way communication are an integral part of the long-term effort. PD (as 
distinct from personal improvement) should have at least two levels of observable, 
measurable, impact: 1) to improve the participant’s observed professional practice when 
rrieasured against stated criteria, and 2) to influence positively the achievement of students 
when measured against desired outcomes. The new professional practice will be sustained 
as part of the professional’s armamentariiun. 

PD, as defined here, excludes planned formal advanced work, such as for a degree 
or to add certification. Experience, a type of on-the-job training (OJT) is also excluded 
here as PD. 

Staff Develonment (SD) involves workshops, training and knowledge related to the 
workplace, and offered to both professional and support personnel. The focus may not be 
on classroom performance but rather on personal and job-related topics of interest and 
value to staff and to organization maintenance or health (e.g., retirement planning, first aid, 
diversity training, conflict resolution, policies/procedures related to law, etc.). 



* These definitions are adapted from operational definitions used in two recent EdD 
dissertations: Covert, (2003) pp. 16-17 and Tienken, (2003) pp. 14-15. In both studies 
the researchers explained the connection of PD to observed changes in teacher behavior. 
Tienken connected the job-embedded work to measured student improvement in writing 
(experimental design); Covert sought to determine if any changes in teacher behavior 
persisted at least a full year (time-series design) in use of running records. 
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Appendix B 

A Myopia Check for Staff Development (SD)* 

Sparks (1995) suggested a need for a “paradigm shift” in SD. Sparks listed three 
forces driving the shift: Results-driven education (learning vs. teaching); systems 
thinking (inter-relatedness); and constuctivism (competing with logical positivism) (p. 

2). “While the knowledge, skills and attitudes of individuals must continually be 
addressed, quality improvement expert W. Edwards Deming estimates that 85% of 
barriers to improvement reside in the organization’s structure and processes, not in the 
performance of individuals” (p. 3). [In later years, Deming (1993, pp. 25-34) adjusted 
this estimate to 94% - 6%.] 

If 85% or more of barriers to improvement are in the organization, and only 15% 
or fewer reside in the performance of individuals, educators might attend to the 
organization, and especially to those elements that have been shown through SBR 
research to have significant effects on student outcomes. Improvements in student 
outcomes include achievement in four areas, the Abecedarian Compact or A, B, C, D’s: 
Academics, Behavior and discipline. Citizenship and participation. Development into 
competent, productive adults. 

To a school administrator, 85%-94% should seem like a better bet for action than 
6%-15%. What, might educators consider as a first step in getting school reform on track 
to make schools better places for kids? A few research-based ideas appear here; others 
are provided in Table 2. 



• Class Size 

• School Size 

• Use of Time 

• Grade Retention 



• Transitions 

• Individualize Instruction 

• Looping, Cohorts 

• Grade Levels 



The 15% is important, of course, because of the labor-intensive nature of 
Education and the need for continuing renewal. Yet, the 85% seems to be the place for 
the intense work. The message of this paper is that we should change the organization, 
the context for education, and de-emphasize the PD tunnel vision, the purpose of which 
seems to blame teachers, sell projects, pay consultants, etc. rather than to improve the 
organization and student outcomes. 



* Those with “Projects” and training to sell will offer vested-interested denials of this, 
position. If SD were 100% effective (wishful-thinking) it could address 15% or less of 
the problem that PD’s advocates claim exists. 
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Knowled.ge Survey of Research-Based Student-Outcome Practices (K-12) 



I’m collecting information for a paper. Data and results will be anonymous. 
Your completion of the survey is completely voluntary. Please return the 
material to the person who distributes it or directly to: C. M. Achilles, 304 
Porter Ed. Bldg., EMU, Ypsilanti, MI 48197. 

A Please list key research-based information or concepts that improve 

student outcomes in schooling that you have been taught directly in any 
of your formal Educational Administration (EA) program courses at the 
Masters, Ed.S., or Ed.D. levels. (Not in-service efforts, or on-the-job 
training.) Criteria for including the information here: 

a - Research based b - Improved student outcomes 

c - Taught m formal EA classes, not in-service sessions 



Elementary Level (K-8 or so) 




Concept 


A 

B 


//(pffisx A/~ Iooo 


C 


ptrm Xocxt- %003 


D 





E 

F 



2. Secondary Level (8-12 or so) 
Concept 

p]odql 

B 

C " ^ 

F 



B. If you were also taught directly HOW to do any of the concepts you 
entered into question 1 or/and 2 above, please mark that item with a 
check (x) in the space at the left of your entry. 

C. Identifier Information 

\ Your FA program level : Masters, Ed.S., Doctoral (ple^e circle one). 

2. State: 3. Years experience in EA A ^ 

Thank you . 

D. If you are interested in the results. 

Complete this leist item. 

N am e: Address: 



Date: 
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A Six-Level Phase Model to Guide Practitioners 

Care is required in selecting options for education policy especially if the research Jot 
education is conducted hy persons from other disciplines that may have criteria for 
acceptance and publication different from those in education. The points made above have 
been combined into a six-level model to help policy persons and practitioners. sort the issues 
into valid and invalid claims based upon reasonable criteria. The serious task of educating 
America’s youth requires more than faked data being called “valid” by the sound-byte media. 

Level I . The base for intelligible research is clarity of definition . Before considering 
using “research” ideas, carefully evaluate the definition of terms to assure that the “research” 
base is accurate. Check its fidelity to the policy or program being considered. All reputable 
research should have a section on definitions, and the research and data should relate 
explicitly to the defined terms. 

Level II . Assess research and results against established criteria, such as Crane 
(1998): a) Benefits outweigh costs? b) Statistically significant effect? c) Magnitude and 
longevity of the effect? d) Relationship of evaluator to the project? e) Replicability? f) 
Maintenance of effects on larger scale, etc.? 

Level III . Assure that the author(s) avoided errors that slant reviews of research. 
Dunkin’s (1996) nine types of errors can be guidelines: 1) Unexplained selectivity, 2) Lack 
of discrimination, 3) Erroneous detailing, 4) Double counting, 5) Non-recognition of faulty, 
author conclusions 6) Unwarranted attributions, 7) Suppressions of contrary findings, 8) 
Consequential errors, and 9) Failure to marshal evidence relevant to a generalization. 

Level rv . Consider possible conflict of interests. Does the author have significant 
monetary or reputational stake in a specific point of view? Does the research rest, at least 
partially, on independent, unbiased work of reasonably disinterested persons? 

Level V . Referee and ideology. Are sources used in the decision process refereed? 
Are the publications from a generally non-biased outlet, as distinct from ideologically driven 
sources? (E.g., Some think tanks). 

Level VI . Synthesize and evaluate the results of Levels I-V within the local context 
where the policy or program is proposed. Does this make sense for you? 

This six-level phase model provides a process to guide a decision. If the policy or 
program fails Level I, the action is over. A policy or program that has high value (potential 
impact) must reach a high level in the model, say Level IV or V if it is to be considered 
seriously. At that point, the leader should visit the primary research before making a final 
decision, and conduct a thorough literature review. (See also Crane, Appendix F, and 
Appendix G that relates STAR research to the elements of Scientifically Based Research or 
SBR). 
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AUTHOR NOTES 



1, C. M. Achilles, EdD is professor of education administration (EdAd) at Seton Hall 
University (SHU) and Eastern Michigan University (EMU) each 50% time. He has 
also taught EdAd since 1967 at the University of Tennessee, Knoxville, University of 
North Carolina at Greensboro, and part time at Nova Southeastern University. He 
was a principal investigator of Tennessee’s STAR class-size study, other class-size 
studies. Project SHAL (an Effective Schools Initiative in St. Louis), and a researcher 
on the Phi Delta Kappa Commission on Confidence in Public Schools. He is author 
or co-author of numerous books, chapters, monographs, articles, research reports and 
ERIC entries. 

Thanks to S. Covert, C. Tienken, M. Caufield-Sloan, Nan-Chi Tiao, and others who 
have helped me track down sources and clarify points made here. Criticisms by D. 
Berliner and J. Finn helped me refine definition and focus. All blame for errors or 
hyperbole rests solely with the author. 

This paper is an adaptation of material submitted for publication review. Please do 
not distribute this paper or quote extensively from it without first contacting the 
author: 4477 Snug harbor, Geneva, NY 14456 (315-789-2399). 
<plato936@rochester.rr.com> If you use e-mail, include your mail address and 
phone contact. 



2. Each person might add favorite NCLB aliases to this list. Entries here include 
suggestions I’ve heard or read, but cannot recall the sources. My apologies. More 
appropriate names than NCLB Act might be: No Child Left Act, Testing is Teaching 
Act, Test Publishers Employment Act, Child Unfunded Mandate Act, Suffer Little 
Children Act, Statistically Impossible Act, Clueless About Education Act, Forget 
Years of Research Act, Pay Campaign Debt Act, etc. 



3. Adger, C. T. (2002, December). Professional conversation and teacher learning. 
Educational Researcher 31 (9), 26-28 reviews two edited volumes published by 
Teachers College Press that emphasize teachers behaving as professionals to improve 
their skills. The review is of Clark’s (2001) Talking Shop: Authentic Conversation 
and Teacher Learning and of Lieberman and Miller’s (2001) Teachers Caught in the 
Action: Professional Development that Matters. The task still is to link examples of 
changed teacher behavior to student outcomes. 
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